Human Evaluation of English–Irish Transformer-Based NMT

In this study, a human evaluation is carried out on how hyperparameter settings impact the quality of Transformer-based Neural Machine Translation (NMT) for low-resourced English–Irish pair. SentencePiece models using both Byte Pair Encoding (BPE) and unigram approaches were appraised. Variations in model architectures included modifying number layers, evaluating optimal heads attention testing various regularisation techniques. The greatest performance improvement was recorded Transformer-optimized with 16k BPE subword model. Compared baseline Recurrent Network (RNN) model, demonstrated BLEU score 7.8 points. When benchmarked against Google Translate, our translation engines significant improvements. Furthermore, quantitative fine-grained manual conducted which compared machine systems. Using Multidimensional Quality Metrics (MQM) error taxonomy, types generated by an RNN-based system explored. Our findings show best-performing Transformer significantly reduces accuracy fluency errors when